same plots as in simulations_v2.Rmd, but added sample sizes to the boxplots

# 2 ts lengths, 2 obs errors, 2 thresh quantiles,2 beta_sd's = 16 combinations, x3 driver response relationships (not including linear)

# number of simulations to run
nsim1 <- 50 # should to 100 but doing less for now so it takes less time
nsimb <- 10 # fewer simulations for bootstrapping

xvals1 <- seq(from = 0, to = 6, by = 0.01)

# parameter combinations
obs_set <- c(0.1, 5)
#obs_set <- c(0.5, 5)
ts_set <- c(15, 45)
quant_set <- c(0.15, 0.5)
#quant_set <- c(0.25, 0.5)
beta_sd_set <- c(0.01, 3)
#beta_sd_set <- c(0.01, 2)
#cov_set <- c(FALSE, TRUE)

parmat <- unname(as.matrix(expand.grid(obs_set, ts_set, quant_set, beta_sd_set)))

#parmat

# control pars for each driver-response relationship
# sigmoid v1
control_pars_sg1 <- list(thresh_loc = 3, thresh_loc_sd = 0, y_max = 3, y_min = 0, hs_type = 1, hs_a = NULL, skew_cv = 1, skew_conc = "up", sig_k = 1.5, lin_m = 1, lin_b = 0)

# skew v1 (concave down)
control_pars_sk1 <- list(thresh_loc = 3, thresh_loc_sd = 0, y_max = 3, y_min = 0, hs_type = 2, hs_a = NULL, skew_cv = 1, skew_conc = "down", sig_k = 1.5, lin_m = 1, lin_b = 0)

# hockey stick v1
control_pars_hs1 <- list(thresh_loc = 3, thresh_loc_sd = 0, y_max = 3, y_min = 0, hs_type = 1, hs_a = NULL, skew_cv = 1, skew_conc = "up", sig_k = 1.5, lin_m = 1, lin_b = 0)

# linear
control_pars_ln1 <- list(thresh_loc = 3, thresh_loc_sd = 0, y_max = 3, y_min = 0, hs_type = 1, hs_a = NULL, skew_cv = 1, skew_conc = "up", sig_k = 1.5, lin_m = 1, lin_b = 0)

compare jackknife and bootstrap (sigmoidal only, nsim = 10)

number of thresholds detected

Plot the difference in the number of thresholds detected each simulation (for bootstrap, this is either 0 or 1, while for jackknife this is the average of the number of thresholds detected each jackknife iteration)

Positive values mean that on average jackknifing detected more thresholds than bootstrapping while negative values mean bootstrapping detected more thresholds than jackknifing

In this plot (and all boxplots below), each panel is a different combination of 1) observation error, 2) time series length, and 3) standard deviation in the effect of the covariate on the response (beta_sd, where beta ~ abs(N(0, beta_sd)) is the slope of the relationship between the covariate and the response). Columns 1 and 3: obs error = 0.1. Columns 2 and 4: obs error = 5. Columns 1-2: time series length = 15. Columns 3-4: time series length = 45. Top row: beta_sd = 0.01. Bottom row: beta_sd = 3.

Within each panel, the results are grouped based on thresh_quant, which is the quantile of the driver data that the threshold falls in (i.e., quant = 0.15 means the true threshold is equal to the 15% quantile of the driver values, and quant = 0.5 means the true threshold is eaual to the median of the driver values), and the subgroupings represent the the method used to calculate the threshold. Purple = local max/min in the second derivative that has the largest absolute value, green = local min in second deriv that is furthest from zero, orange = location where the first derivative crosses zero (a max/min in the response), light blue = location where the second derivative crosses zero (an inflection point)

magnitude of difference from true threshold

Plot the difference in the absolute value of the difference between the threshold estimate and the true threshold value, i.e., abs(jack_est - true) - abs(boot_est - true)

Positive values mean the bootstrapped estimate was closer to the true value, i.e., abs(jack_est - true) > abs(boot_est - true), while negative values mean the jackknifed estimate was closer.

jackknifing results (nsim = 50, sigmoidal, skew, and hockeystick relationships)

average number of thresholds detected

Plot the average number of thresholds detected across the jackknife iterations each simulation

sigmoidal relationship

skew relationship

hockeystick relationship

difference between mean estimate and true threshold

Plot the difference between the mean threshold estimate (i.e., average of the estimates across jackknife iterations in a simulation) and the true value of the threshold

sigmoidal relationship

skew relationship

hockeystick relationship

RMSE of estimate

Plot the RMSE of the threshold estimates from the jackknifing each simulation

sigmoidal relationship

skew relationship

hockeystick relationship